Comparative Evaluations in the Domain of Automatic Speech Recognition

Abstract The goal of this contribution is threefold: (1) to present the results of a comparative evaluation of different, academic and commercial, speech recognitions engines; (2) to study relative performances of Hidden Markov Model and hybrid technologies, as used in stateof-the-art systems; and (3) to study the impact of different linguistic resources, such as simple word spotting, statistical and grammarbased language models, on the speech recognition accuracy. All the evaluations were made on the basis of the same test data sets and conclusions derived from the obtained Word Error Rate scores. The evaluated speech recognition engines are all speaker independent, continuous speech recognition engines, either academic systems widely used in the research community or commercial tools currently available on the market. In this work, we considered three academic systems (HTK, Sirocco, and Strut/DRSpeech) and two commercial ones (for the confidence reasons, we name these systems SRE1 and SRE2). The main obtained results are that (1) the Hidden Markov Model (HMM) based technology performs better than the hybrid approach in the case of unconstrained continuous speech, and (2) the academic systems perform better in the case of continuous speech in French, while the commercial systems show better recognition accuracy for continuous speech in German.